From pre-test and post-test probabilities to medical decision making

Background A central goal of modern evidence-based medicine is the development of simple and easy to use tools that help clinicians integrate quantitative information into medical decision-making. The Bayesian Pre-test/Post-test Probability (BPP) framework is arguably the most well known of such tools and provides a formal approach to quantify diagnostic uncertainty given the result of a medical test or the presence of a clinical sign. Yet, clinical decision-making goes beyond quantifying diagnostic uncertainty and requires that that uncertainty be balanced against the various costs and benefits associated with each possible decision. Despite increasing attention in recent years, simple and flexible approaches to quantitative clinical decision-making have remained elusive. Methods We extend the BPP framework using concepts of Bayesian Decision Theory. By integrating cost, we can expand the BPP framework to allow for clinical decision-making. Results We develop a simple quantitative framework for binary clinical decisions (e.g., action/inaction, treat/no-treat, test/no-test). Let p be the pre-test or post-test probability that a patient has disease. We show that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$r^{*}=(1-p)/p$$\end{document}r∗=(1-p)/p represents a critical value called a decision boundary. In terms of the relative cost of under- to over-acting, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$r^{*}$$\end{document}r∗ represents the critical value at which action and inaction are equally optimal. We demonstrate how this decision boundary can be used at the bedside through case studies and as a research tool through a reanalysis of a recent study which found widespread misestimation of pre-test and post-test probabilities among clinicians. Conclusions Our approach is so simple that it should be thought of as a core, yet previously overlooked, part of the BPP framework. Unlike prior approaches to quantitative clinical decision-making, our approach requires little more than a hand-held calculator, is applicable in almost any setting where the BPP framework can be used, and excels in situations where the costs and benefits associated with a particular decision are patient-specific and difficult to quantify. Supplementary Information The online version contains supplementary material available at 10.1186/s12911-024-02610-3.

Since neither the cost of over-action or under-action is negative, rearrangement yields: r * < r. (5) Case 2: If c ii − c aa < 0 and c ia − c ai > 0, inaction is always optimal.The constraint implies c aa > c ii and c ia > c ai .Thus, both costs associated with action are larger than the costs associated with inaction, so inaction is always optimal.
Case 3: If c ii − c aa > 0 and c ia − c ai < 0, action is always optimal.The constraint implies c ii > c aa and c ai > c ia .Thus, both costs associated with inaction are larger than the costs associated with action, so action is always optimal.
Case 4: If c ii − c aa < 0 and c ia − c ai < 0, it is optimal to act if r < r * .Using a similar argument as in Case 1, we will act if: However, since the cost of over-action is negative, the sign on the equality flips when rearranging: (1 − p)(c ia − c ai ) < p(c ii − c aa ) (7)

Breast Cancer Case Study
This case study concerns genetic testing for breast cancer.We demonstrate how to use a cost function based on one element (e.g., mortality) as a benchmark to guide decisions.In Section 2.1, we present the entire case study, and Section 2.2 comprises all supporting mathematical details.

Prophylactic Mastectomy for a BRCA Carrier
Breast cancer is responsible for over 464,000 deaths per year globally [1].Genetically, BRCA1 and BRCA2 gene mutations are responsible for 80% of all hereditary breast cancers and 5 to 6% of all cases of breast cancer [2].Carriers of BRCA1 and BRCA2 have a 40 to 80% lifetime risk of breast cancer [3], much higher than the 8 to 10% risk for the general population [4].Because of that risk, such patients may consider prophylactic bilateral mastectomy [5].For many patients, this tremendously difficult decision problem involves a myriad of objective and subjective factors, including mortality, morbidity, psycho-social, and financial costs such as procedural costs and loss of income.
Consider an otherwise healthy 30 year old mother of two with an extensive family history of breast cancer (see   As in the prior bacteriuria case-study, these are just two out of an infinite number of possible ways to present and evaluate this decision boundary.In fact, we expect that some readers will object to our second presentation: as most humans will not live 100 years, considering costs as 104 years of life lost may distort the problem.As in the prior case study, this case-study merely illustrates how extended BPP framework can be a quantitative framework for medical decision-making and highlights ways to use it.Moreover, this case-study illustrates how to incorporate quantified measures of costs with the SPK framework, simplifying the decision problem to the remaining unquantified factors.

Calculating Post-Test Probability
Highly sensitive and specific tests exist for testing for BRCA.In the case of BRCA1, these tests have a reported average sensitivity of 97.1% (95%-CI: 95.2% -98.5%) and specificity of 100% (95%-CI: 96% -100%) [6].To be conservative, we assume the midpoint of each range in our calculations, resulting in a sensitivity of 96.85% and a specificity of 98%.These percentages result in a positive likelihood ratio of 48.4 (LR + ) and a negative likelihood ratio of 0.03 (LR − ).Although some tests have no reported false positives (i.e., a specificity of 100%), we do not assume which test was used and, therefore, use a more conservative estimate.
We considered an otherwise healthy 30 year old woman with an extensive familial history of breast cancer (see Table 1) and no family history of ovarian cancer.Every incidence of breast cancer involved a single breast, and each family member was white and not an Ashkenazi Jew.Estimated pre-test probabilities for our case study were computed using the BayesMendel package in R [7].
Based on her family history, BayesMendel returns a 33% pre-test probability of being a BRCA1 carrier (pre-test odds: 0.493).This probability, along with the positive LR above, results in a post-test probability that she is a BRCA1 carrier of 95.9% (post-test odds: 23.839).

Estimating Years of Life Lost
We used a mortality-based metric, the expected years of life lost, in our breast cancer example.We computed it directly from available life expectancy tables in the diseased and non-diseased population.
Consider a hypothetical, currently healthy patient at age a.This patient may or may not be diagnosed with breast cancer and may or may not die of breast cancer if diagnosed.Let t denote the age at diagnosis and s denote the age at death if diagnosed.
Following the basic properties of expectation, we write the expected years of life lost as the expected difference in the life expectancy for the average person at age a (LE a ) and the age at death for those that are diagnosed: This formulation allows us to incorporate uncertainty in the age at diagnosis into the calculation (as seen in the parenthesis of Equation 9).For BRCA carriers, this metric can be readily calculated from pre-existing data, including the general population life expectancy tables [8], age at diagnosis data for BRCA carriers [9], and death rates following years since diagnosis for breast cancer [10].
We can calculate an analogous measure for expected years of life lost if treatment is pursued but the person is not diseased (over-treatment).In our case study, this is the expected years of life lost if a person pursues a prophylactic mastectomy but does not carry a BRCA mutation, i.e. the death rate for the preventative surgery.We obtained data for this calculation from El-Tamer et al. [11].
We make several assumptions: diagnoses, if they occur, happen before the age of 80 (as noted on the bounds of integration in the parenthesis in Equation 9); the maximum lifespan is 100 years old (as noted on the bounds of integration outside of the parenthesis); and survival for more than 10 years after diagnosis constitutes a "cure" from breast cancer.
We used Monte Carlo simulation to estimate these measures.The general procedure to estimate years of life lost to breast cancer in a BRCA carrier who forgoes an prophylactic mastectomy is in Algorithm 1.For each estimate, we used 1,000 samples.
Algorithm 1 Estimating years of life lost due to breast cancer for BRCA carriers who do not seek preventative treatment.Draw an age at death for the general population, LE i ∼ f general pop 18: Calculate the expected years of life lost, Y LL i = LE i − s i 19: end for 20: Return mean(Y LL) 3 Additional Details for Re-Analyzing Morgan et al.

Deriving Implied Cost Ratio
We show that the implied cost ratio in our re-analysis of Morgan et al. [12] is r = o act /o disease .This result assumes that the odds that a physician will act is proportional to their perception of the expected costs: Combining the above ratio for r with the assumption above, we obtain which, using the relationship between probability and odds, implies: Recall that the expected cost of not treating is (1 − p disease )c ai + p disease c ii , and the expected cost of treating is p disease c aa + (1 − p disease )c ia .Under the assumption that the cost of the correct decision is minimal (i.e., c aa ≈ 0 and c ai ≈ 0), this is exactly the equation presented in the main text as it cancels with the second term on the right-hand side of the equation above:

Exclusion of the UTI Survey Results
In our re-analysis of Morgan et al. [12], we disregard their UTI case study, due to both survey design and corresponding cited literature.The presented clinical scenario was as follows: "Mr.Williams, a 65-year-old man, comes to the office for follow up of his osteoarthritis.He has noted foul-smelling urine and no pain or difficulty with urination.A urine dipstick shows trace blood.He has no particular preference for testing and wants your advice." In this scenario, the patient presented with both blood in the urine and foulsmelling urine, yet the researchers assumed asymptomatic bacteriuria.The predictive power of these symptoms are often noted as controversial or misleading in the medical literature (e.g., Midthun et al. [13], Jump et al. [14]), yet around half of family practice physicians consider them when considering antibiotic therapy for UTIs [15].Indirectly, the discussion of Morgan et al. notes this as the differences observed in this example "may reflect the evolution of the definition of asymptomatic bacteriuria as a separate entity from UTI" [12].Thus, we exclude the asymptomatic bacteriuria example from our reanalysis of Morgan et al. [12].

More Notes on the Breast Cancer Survey Cost-Ratio
Though the clinical scenario is somewhat vague with a positive mammogram but no mention of the size or differential diagnosis, the next logical action is to pursue a confirmatory diagnosis with a core needle biopsy.We are unaware of studies studying the mortality risk associated with not doing a biopsy in a patient with cancer (e.g., under-treating); we assume that if this patient had cancer but did not receive a biopsy, she would also not undergo surgery.Prior work studying mortality risk in patients who have breast cancer but refuse surgery (under-treating) suggest that the 5 year mortality risk is approximately 30% [16].In contrast, over-treating consists of mortality risk associated with a core-needle biopsy combined with mortality risk of developing a new breast-cancer after a false-positive mammogram.Since the mortality risk associated with a core-needle biopsy is minuscule, we focus on 5-year mortality risk from breastcancer in patients after receiving a false-positive mammogram.For women who do not have breast cancer, less than 1% are expected to develop breast-cancer within the next 5-years [17].Thus, we have an absolute upper bound of a 1% 5-year mortality rate associated with developing breast cancer in the setting of over-treating.Thus, the cost-ratio for this patient should be at least CR = 0.30/0.01= 30, which is above the implied cost-ratio of 17.In short, these results do not disprove our hypothesis that bluntness of the survey tool contributed to the physicians' elevated probability estimates.
3.4 Calculating Cost Ratio using Heckerling et al.
In Heckerling et al. [18], 52 physicians reviewed the charts of their own patients who had presented to an emergency department with fever or respiratory complaints.They

Table
we can at least simplify the problem and create a modified decision boundary to evaluate by only considering the remaining unquantified factors (e.g., morbidity, psycho-social, and monetary).Working with inverse cost-ratios again, we have 1/r mortality = 1/6.56= 0.152, far less than 1/r Alternatively, we investigate how much we need to modify the mortality cost-ratio to warrant treatment with our original boundary.We can formulate this calculation For each additional year of life lost due to under-treating, other factors must contribute an additional 23.8 years of life lost to over-treatment to warrant inaction.
).Based on her genetic test results and her family history, she has an approximately 95.9% post-test probability of being a BRCA1 carrier (post-test odds of 23.8; see Section 2.2.1 for further details).Although the BRCA1 genetic test is highly sensitive and specific, we assume some diagnostic uncertainty remains.Based on prior research, objective measures can capture this patient's mortality risks.In Section 2.2.2, we use prior * .The comparison implies that if mortality was the only important factor, we should clearly act.We can now consider the other three factors and ask whether, taken together, they provide a strong enough argument for inaction to overcome the mortality cost.We have two potential approaches for this calculation: modifying the decision boundary, or modifying the mortality cost-ratio.We modify the decision boundary and create a new boundaryr * modified = 1/r * 1/r mortality ≈ 157.This boundary means: excluding mortality costs, morbidity, psycho-social, and monetary costs must imply that over-treating is at least 157 times more costly than under-treating to warrant inaction.Again, mental tricks may help evaluate this modified boundary.As in the prior case-study, this boundary can be phrased in monetary terms: excluding the mortality cost, if under-treating a patient costs $1000, would you pay at least $157,000 to avoid over-treating?

Table 1
The example patient's family history of breast cancer.

1 :
Data: Current age of patient: a 2: Number of Monte Carlo samples: n samps 3: Result: Expected years of life lost: E(Y LL) 4: for i = 1, ..., n samps do Set age at death s i = t i + δ i Declare a "cure" and draw the age at death from the general population, i.e. s i ∼ f general pop Draw the age at death from the general population, s i ∼ f general pop